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Mail Stop Issue Fee 
Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 

Sir: 

Claims 1, 4-10, 13-19 and 22-38 of the above-identified application were allowed on June 
3, 2005. In response to the Examiner's Statement of Reasons for Allowance, Applicants submit the 
following comments. 

Applicants initially note that the Examiner omitted certain meaningful language from his 
Statement of Reasons for Allowance. Presented below are the Examiner's statements incorporating 
the omitted claim language by underline that the Applicants believe should have been provided. 
Also, the "strike-through" text should be deleted. 

The amendment filed 2/24/05, in light of applicants' comments, places the application into 
condition for allowance by incorporating into all of the independent claims that the intent 
determination comprises resolving referential ambiguity associated with the one or more 
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userfs) and the one or more device(s) in the environment based on at least a portion of the 
received multi-modal data. 

4 amended - system wherein the execution of the one or more actions in the environment 

eff e cts the comprises controlling at least one of the one or more devices in the environment 
to at least one of effectuate the determined intent, effect the determined focus, and effect the 
determined mood of the one or more users. 

5 amended - system , . . . wherein the execution of the one or more actions c o ntrols in the 
environment comprises controlling at least one of the one or more devices in the environment 
to request further user input to assist in making at least one of the determinations. 

6 amended - system wherein the execution of the one or more actions initiates comprises 

initiating a process to at least one of further complete, or correct^ or and disambiguate what 
the system understands from previous input. 

7 amended - system wherein the at least one processor is further configured to abstracts 

the received multi-modal input data into one or more events prior to making the one or more 
determinations. 

8 amended - system wherein the at least one processor is further configured to performs 

one or more recognition operations on the received multi-modal input data prior to making 
the one or more determinations. 

9 amended - system with f e atu re s of both claims 7 amended and 8 am e nd e d . 10 amended = 
m e thod c o rr e sponding t o claim 1 amend e d . 13 amended - method c o rr e sponding t o claim 4 
am e nded . 14 amended - method corr e sp o nding to claim 5 am e nd e d . 15 amended - meth o d 
c o rr e sponding to claim 6 am e nd e d . 16 amended - m e th o d c o rr e sponding to claim 7 amended . 
17 amended - m e thod corr e sponding t o claim 7 amend e d . 18 amended - machine readable 
medium corr e sp o nding to claim 1 amend e d . 19 amended - system like claim 1 am e nded and 
sp e cifying . . . including at least the audio -based data and imag e-based data. 

29 amended - system like claim 19 amended wherein the execution of the one or more 
actions controls in the environment comprises controlling at least one of the one or more 
devices in the environment to e ffects at least one of effectuate the determined intent, effect 
the determined focus, and effect the determined mood of the one or more users. 

30 amended - system like claim 19 amended wherein the execution of the one or 
more actions con t rols in the environment comprises controlling at least one of the one or 
more devices in the environment to request further user input to assist in making at least 
one of the determinations. 
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31 amended - system like claim 19 amended wherein the at least one processor is further 
configured to abstracts the received multi-modal input data into one or more events prior to 
making the one or more determinations. 

32 amended - system like claim 19 amended wherein the at least one processor is further 
configured to performs one or more recognition operations on the received multi-modal input 
data prior to making the one or more determinations. 

36 amended - system like claim 19 amended wherein the execution of the one or more 
actions comprises initiating ini t iat e s a process to at least one of further complete a or correct^ 
and or disambiguate what the system understands from previous input. 

37 amended - system with fea t ures of both claims 31 am e nd e d and 32 amended . 

38 amended - m e th o d corresp o nding t o claim 19 am e nd e d . 

The language as edited above is consistent with the allowed claims. Applicants respectfully 
request that the Examiner's Statement of Reasons for Allowance be revised in accordance with the 
language as set forth above. 



Respectfully submitted, 



Date: June 16, 2005 



Attorney for Applicants ( 
Reg. No. 39,274 
Ryan, Mason & Lewis, LLP 
90 Forest Avenue 
Locust Valley, NY 11560 
(516) 759-2946 
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